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International application No. PCT/G BOO/04635 



I. Basis of the report 

1 . With regard to the elements of the international application (Replacement sheets which have been furnished to 
the receiving Office in response to an invitation under Article 14 are referred to in this report as "originally filed" 
and are not annexed to this report since they do not contain amendments (Rules 70, 16 and 70. 1 7)): 
Description, pages: 

6-26 as originally filed 

1-5 as received on 26/02/2002 with letter of 25/02/2002 
Claims, No.: 

1-20 as received on 07/03/2002 with letter of 06/03/2002 
Drawings, sheets: 

1/21-21/21 as originally filed 

2. With regard to the language, alt the elements marked above were available or fumished to this Authority in the 
language in which the intemational application was filed, unless otherwise indicated under this item. 

These elements were available or furnished to this Authority in the following language: , which is: 

□ the language of a translation furnished for the purposes of the international search (under Rule 23.1(b)). 

□ the language of publication of the international application (under Rule 48.3(b)). 

□ the language of a translation furnished for the purposes of international preliminary examination (under Rule 
55.2 and/or 55.3). 

3. With regard to any nucleotide and/or amino acid sequence disclosed in the international application, the 
international preliminary examination was carried out on the basis of the sequence listing: 

□ contained in the international application in written form. 

□ filed together with the international application in computer readable form. 

□ furnished subsequently to this Authority in written form. 

□ furnished subsequently to this Authority in computer readable form. 

□ The statement that the subsequently furnished written sequence listing does not go beyond the disclosure in 
the international application as filed has been furnished. 

□ The statement that the information recorded in computer readable form is identical to the written sequence 
listing has been furnished. 

4. The amendments have resulted in the cancellation of: 
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□ 



the description, 
the claims, 
the drawings. 



pages: 



□ 



Nos.: 



□ 



sheets: 



5. H This report has been established as if (some of) the amendments had not been made, since they have been 

considered to go beyond the disclosure as filed (Rule 70.2(c)): 

(Any replacement sheet containing such amendments must be referred to under item 1 and annexed to this 
report.) 

see separate sheet 

6. Additional observations, if necessary: 



III. Non-establishment of opinion with regard to novelty, inventive step and industrial applicability 

1 . The questions whether the claimed invention appears to be novel, to involve an inventive step (to be non- 
obvious), or to be industrially applicable have not been examined in respect of: 

□ the entire international application. 
H claims Nos. 35,36. 



□ the said international application, or the said claims Nos. relate to the following subject matter which does 
not require an international preliminary examination (specify): 



□ the description, claims or drawings {indicate particular elements below) or said claims Nos. are so unclear 
that no meaningful opinion could be formed (specify): 



□ the claims, or said claims Nos. are so inadequately supported by the description that no meaningful opinion 
could be formed. 

S no international search report has been established for the said claims Nos. 35,36. 

2. A meaningful international preliminary examination cannot be carried out due to the failure of the nucleotide 
and/or amino acid sequence listing to comply with the standard provided for in Annex C of the Administrative 
Instructions: 

□ the written form has not been furnished or does not comply with the standard. 

□ the computer readable form has not been furnished or does not comply with the standard. 



IV. Lack of unity of invention 

1 . In response to the invitation to restrict or pay additional fees the applicant has: 



because: 
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K restricted the claims. 

□ paid additional fees. 

□ paid additional fees under protest. 

□ neither restricted nor paid additional fees. 

2. □ This Authority found that the requirement of unity of invention is not complied and chose, according to Rule 

68.1 , not to invite the applicant to restrict or pay additional fees. 

3. This Authority considers that the requirement of unity of invention in accordance with Rules 13.1. 13.2 and 13.3 is 

□ complied with. 

□ not complied with for the following reasons: 

4. Consequently, the following parts of the international application were the subject of international preliminary 
examination in establishing this report: 

□ all parts. 

H the parts relating to claims Nos. 10-15,17,18,22-29,32-34. 

V. Reasoned statement under Article 35(2) with regard to novelty, inventive step or industrial applicability; 
citations and explanations supporting such statement 



1. Statement 



Novelty (N) 



Yes: 
No: 



Claims 10-15.17,18,22-29, 33, 34 
Claims 32 



Inventive step (IS) 



Yes: 
No: 



Claims 1 0-1 5, 1 7, 1 8,22-29 
Claims 32-34 



Industrial applicability (lA) 



Yes: 
No: 



Claims 1 0-1 5, 1 7,1 8,22-29,32-34 
Claims 



2. Citations and explanations 
see separate sheet 
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Preamble: 

Reference is nnade to the following documents: 

D1 : WO 93 08540 A (STANFORD RES INST INT) 29 April 1 993 (1 993-04-29) 
D2: DE 41 43 364 A (KOLLER ROMAN) 30 September 1993 (1993-09-30) 
D3: EP-A-0 474 232 (SHARP KK) 1 1 March 1992 (1992-03-1 1) 

Re Item I. 

1 . With the letter dated 06.03.2002 (received on 07.03.2002), the Applicant filed 
completely new claims 1-20 which are directed to an apparatus for processing a 
signal to determine a spatial position (cf. independent claim 1) and to a method of 
signal processing (cf. independent claim 11). The new claims have been filed 
without an indication of the basis for the amendments and said claims do not 
correspond to any claims in the original set of claims (claims 1-36). It is 
furthermore not apparent that there is a basis for these claims in the original 
figures or in the original description. 

Therefore, claims 1-20 filed with the letter dated 06.02.2002 introduce subject- 
matter which extends beyond the content of the application as filed, contrary to 
Article 34(2)(b) POT. 

Hence, this report has been established as if the amendments to claims 1-20, filed 
with letter of 06.03.2002, had not been made. Accordingly, the observation made 
under Items III., IV. and V. refer to claims 1-36 of the application as originally filed. 

Re Item VI. 

This International Examining Authority found multiple (groups of) inventions in this 
international application, as follows: 

1. Claims: 1-4,16,21 

input device with touch and orientation sensing with respect to the ambient 
magnetic field 
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2. Claims: 5-9 

input device orientation sensing with respect to gravity 

3. Claims: 10-15,17,18,22-29,32-34 

input device with sound actuated touch detectors 

4. Claim: 19 

input device connected to a computer switching between text and graphical mode 

5. Claim : 20 

input device with rechargeable cell 

6. Claims: 30,31 

spherical touch input device mapped to a three dimensional object 

The prior art document D1 discloses a substantially spherical input device including 
touch-sensing means for generating position signals indicative of the position of touch 
events occurring on the surface of the object, electro-magnetically based orientation 
sensing means, and means for combining touch and orientation Input data for 
transmission to a computer (cf. abstract; figure 1 and associated description). 

This document therefore covers the inventive idea of independent claims 1,16 and 21 
and there is no STF (Special Technical Feature), as defined in Rule 13.2 PCT in these 
independent claims. The inventive idea of dependent claim 2 is also covered by the 
prior art document and there is no STF. The STF of the first subject can be found in 
claims 3 and 4, i.e. to include ambient magnetic field sensors, solving the objective 
problem to identify the rotation of the device about the magnetic north axis (cf . Fig. 1 9, 
step 1901). 

The STF of subject 2 (depending claims 5 to 9) is that the orientation is sensed with 
respect to gravity using a mobile inductor core and three sensing cores. 
The objective problem is to identify rotation of the device about the vertical axis (cf. Fig. 
19, step 1903). 

The STF of subject 3 (depending claims 10 to 15 and 22 to 29 and independent sets of 
claims 32 to 34) is that the sound produced by an object touching the surface is 
sensed. 

The the objective problem solved by these claims is to detect the movement of a finger 
across the sphere (cf. page 8, lines 4-8). 
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The STF of subject 4 (depending claim 19) is that the computer is able to switch 
between graphical and text mode. 

The objective problem is to use the input device for text and graphic input. 

The STF of subject 5 (depending claim 20) is that there is an inductive loop for 
recharging the cell of the input device. The objective problem is to provide wireless 
charging of the internal power supply. 

The STF of subject 6 (independent claims 30 and 31) is that a mapping is generated 

from the input sphere to a three-dimensional object model. 

The objective problem is to manipulate three-dimensional objects. 

The above analysis shows that the STFs of the different groups of inventions are not 
the same. A comparison of the objective problems related to the different groups of 
inventions all seen in the light of the description and drawings of the application shows 
that they are all different and have no corresponding technical effects. Consequently 
the STFs of the different inventions do not correspond and the requirements of Unity of 
Invention (Rule 13.2 PCT) are not fulfilled. 

Re Item V. 

1 . The prior art document D1 discloses a substantially spherical input device 
including touch-sensing means for generating position signals indicative of the 
position of touch events occurring on the surface of the object, electro- 
magnetically based orientation sensing means, and means for combining touch 
and orientation input data for transmission to a computer (cf. abstract; figure 1 
and associated description). 

This document therefore covers the inventive idea of independent device claims 1 
and 16 and the corresponding method claim 21 . Hence, said claims do not meet 
the requirements of Article 33 PCT. 

2. Dependent claim 10 adds to claim 1 the feature that said sensing means 
comprises a plurality of sound transducers. Similarly, dependent claim 17 adds to 
claim 16 that said touch sensing means comprises a plurality of microphones and 
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dependent claim 22 adds to claim 21 the feature that said sensing means includes 
acoustic transducer means. Corresponding features are not disclosed or 
suggested by D1. 

Hence, the objective problem solved by these claims is to detect the movement of 
a finger across the sphere (cf. page 8, lines 4-8). 

Documents D2 or D3 teach the use of sound transducers or microphones in touch 
sensing devices (cf. D2, column 1 , line 67 - column 2, line 1; D3, page 5, lines 
49/50). However, the teachings of both documents do not appear to lead in an 
obvious manner to the present invention as defined by claims 10, 17 or 22. The 
most important reason is that both documents exclusively deal with touch sensing 
on simple two-dimensional rectangular surfaces and do not contain any hints that 
this general sensing principle could be applied in the much more complex 
situation of a 3-dimensional surface. D2 furthermore teaches that a specific 
surface structure has to be provided in order to get acoustical signals which can 
be used to determine positional information (cf. figures la/c and associated 
description: "Riffellinien/Stoppelprofil"). Such specific surface structures cannot be 
easily provided on a spherical object like the present input apparatus. As regards 
D3, it has to be noted that the whole mathematical treatment of the position 
calculation requires an in-plane arrangement of the touch surface and all of the 
sensors so that none of the equations given in D3 is applicable in a 3-dimensional 
case. 

Therefore, claims 10, 17 and 22 are considered to comprise novel and inventive 
subject-matter. 

The subject-matter of claims 11-15, 18-20 and 23-29 is novel and inventive since 
these claims depended on one of claims 10, 17 or 22. 

3. Independent claims 32 and 34 are essentially directed to a method of supplying 
positional information to a computer by detecting sound generated on a touched 
surface by acoustic transducer means. Their subject-matter appears to be at least 
clearly suggested by each of documents D2 (cf. column 1 , line 35 - column 3, line 
63) or D3 (cf. abstract and page 5, lines 49/50). 
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Therefore, independent claims 32 and 34 (as well as dependent clainn 33) do not 
meet the requirements of Article 33 PCT. 
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Processing Signals to Determine Spatial Positions 

The present invention relates to an apparatus for processing a signal 
to determine a spatial position, comprising signal input means, processing 
5 means and the memory means, wherein said processing means receives 
input signals from said input means and produces an output signal indicative 
of a spatial position by executing instructions read from said memory means. 
The present invention also relates to a method of signal processing, in which 
an interaction at a spatial position produces an input signal, and said signal is 

10 processed to produce an output signal indicative of said spatial position. 

Apparatus for processing a signal to determine a spatial position is 
described in DE 4 143 364 in which the amplitude of signals derived from a 
microphone are analysed. As the amplitude of a signal increases, it is 
possible to process the signal to determine that a contact position is moving 

15 towards the microphone. A bearing on the movement may be obtained by 
combining the outputs from several microphones and a high level of 
resolution may be obtained by counting the number of impacts made with the 
relief pattern created upon a surface. In a more complex mode of analysis, 
relevant when a contact stylus is moving more quickly, sidebands are 

20 processed that are created by mixing the natural resonance of the stylus with 
the impact rate of the relief pattern. 

A transformation of the signal produced is performed to the extent that 
the sample amplitudes are considered. Further transfonnations are 
performed when looking at sidebands, that may be considered as perfonning 

25 an integration. However, the transformed signal only gives information that 



AMENDED SHEET 



26-02-2002 * 



GB000463: 



2 

can be used in a linear equation to obtain position. Consequently, this limits 
the application of the procedure and creates a further requirement to the 
effect that a surface requires a particular relief pattern is required. 

A photocopier control panel is described in EP 0 474 232 that utilises 
5 time delay of sound signals created by touching between at least three 
microphones. At least two microphones are required to obtain any data at all 
given that the delay of anival is given by subtracting the absolute times. 

A problem with these known techniques is that they rely upon 
substantially linear signal processing and as such their application to spatial 

10 position detection upon a surface is limited. 

According to a first aspect of the present invention, there is provided 
apparatus of the aforesaid type, characterised in that the memory means 
stores a plurality of signal templates; the processing means processes an 
input signal to produce transform data that conveys a characteristic of an 

15 input signal that is con^elated to an active spatial position; the processing 
means compares the transform data with stored templates to produce a 
similarity score for each of the templates; and the processing means 
interpolates a plurality of the similarity scores to produce the output data 
identifying the active spatial position. 

20 Thus, the present invention addresses the problem of non-linearities 

with respect to the surface qualities. It does not require additional aides such 
as the provision of a relief pattern on the surface. Consequently, the inclusion 
of templates allows any surface system to be modelled irrespective of its 
inherent non-linearity. Furthermore, the inclusion of non-linearities of this type 

25 create particular problems when attempts are being made to track 
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movement 

In a prefenred embodiment, the apparatus includes a plurality of 
transducer means wherein each of said transducer means is configured to 
produce an input signal and steps performed by said processing means are 
5 repeated to identify a plurality of active spatial positions. 

The invention will now be described by way of example only, with 
reference to the accompanying drawings, of which: 

Figure 1 shows a sensor and a computer tenminal; 

Figures 2 and 3 detail constmction of the sensor shown in Figure t, 
10 including microphones and a sensor core; 

Figure 4 details components of the sensor core shown in Figure 3, 
including analogue to digital converters, orientation sensors and a digital 
signal processor. 

Figure 5 details the digital signal processor shown in Figure 4\ 
15 Figure 6 details one of the analogue to digital converters shown in 

Figure 4\ 

Figure 7 details a frequency domain analysis of audio data recorded 
from one of the microphones shown in Figure 2, in response to a first type of 
touch event; 

20 Figure 8 details a time domain analysis of audio data recorded from 

two of the microphones shown in Figure 2, in response to a second type of 
touch event; 

Figure 9 details steps executed on the digital signal processor shown 
in Figure 4. including a step of identifying a drag position, a step of identifying 
25 a hit position and a step of identifying sphere orientation; 
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Figure 10 details the step of identifying a drag position shown in 
Figure 9, including a step of correlating with templates and a step of 
identifying intersecting arcs; 

Figure 1 1 details tennplates that are used in the step of correlating with 
templates shown in Figure 10\ 

Figure 12 illustrates the step of identifying intersecting arcs shown in 
Figure 10\ 

Figure 13 details the step of identifying a hit position shown in Figure 

9: 

Figure 14 details the orientation sensors shown in Figure 4, including 
a magnetic field sensor and a gravitational field sensor; 

Figures 15 and 16 detail the magnetic field sensor shown in Figure 14] 
Figures 17 and 18 detail the gravitational field sensor shown in Figure 

14\ 

Figure 19 details the step of identifying sphere orientation shown in 
Figure 9\ 

Figures 20 and 21 detail constnjction of a charger and receiver unit for 
use with the sensor and computer terminal shown in Figure 1\ 

Figure 22 details components of the computer terminal shown in 
Figure 1, including a memory; 

Figure 23 details contents of the computer memory shown in Figure 

22: 

Figure 24 summarises steps performed by the computer terminal 
shown in Figure 1, including a step of calibrating the sensor and a step of 
using the sensor; 
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Figure 25 details the step of calibrating the sensor shown in Figure 24; 
Figure 26 details the step of using the sensor shown in Figure 24, 
including a step of processing touch event data; and 

Figure 27 details the step of processing touch event data shown in 

5 Figure 26. 

A computer terminal is illustrated in Figure 1. The computer terminal 
101 comprises a high resolution display panel 102 and standard personal 
computer circuitry. The display 102 is the only visible part of the computer. 
The components of the computer are built in to the display housing. The 

10 computer is connected to the internet, and provides access to media of 
many different types, including audio, video, applications such as word 
processors, and so on. This highly varied functionality is provided by the 
combination of an internet browser software application and a graphical 
user interface environment such as X-Windows. This combination of 

15 technologies, both hardware and software, with information presented 
through a graphical display, may be considered as a first kind of computer 
generated environment. A second kind of computer generated environment 
is one in which a three dimensional virtual world is presented on a two 
dimensional screen, and various forms of user input enable the user to 

20 navigate, as if physically present, in the virtual space. Virtual spaces of this 
type are presently used widely in games, but also for serious business 
applications, such as the representation of bug tracking in complex 
computer software development. Three dimensional virtual worlds are 
widely believed to be the future of the Internet, though at the present time, 

25 specific interpretations of this vision have not been sufficiently developed to 
provide practical engineering proposals. 
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Claims 

1. Apparatus for processing a signal to determine a spatial 
position, comprising 

5 signal input means (401, 402). processing means (507). and 

memory means (504. 505, 506), wherein said processing means receives 
input signals from said input means and produces an output signal 
indicative of a spatial position by executing instructions read from said 
memory means, characterised in that: 
10 said memory means (504) stores a plurality of signal templates; 

said processing means (507) processes an input signal to produce 
transformed data that conveys a characteristic of said input signal that is 
correlated to an active spatial position; 

said processing means (507) compares said transformed data with 
15 said stored templates to produce a similarity score for each of said 
templates; and 

said processing means interpolates a plurality of selected similarity 
scores to produce said output data identifying said active spatial position. 

20 2. Apparatus according to claim 1, including a plurality of 

transducer means wherein each of said transducer means is configured to 
produce an input signal, and steps performed by said processing means 
are repeated to identify a plurality of active spatial positions. 
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3. Apparatus according to claim 2, wherein said processing 
nneans combine said plurality of spatial positions to identify a location of a 
common source. 

4. Apparatus according to claim 1, wherein said processing 
means processes an input signal to produce said transformed data by 
identifying characteristic frequency-related components of an active input 
signal. 

5- Apparatus according to claim 1. including surface means, 
wherein said signal input means includes acoustic transducer means and 
said acoustic transducer means are arranged to generate active input 
signals in response to movements made against said surface means. 

6. Apparatus according to claim 5. wherein said surface means 
is finger-operable. 



7. Apparatus according to claim 6, wherein said surface means 
20 is curved. 

8. Apparatus according to claim 7, wherein said curved surface 
is fully enclosed. 

25 9. Apparatus according to claim 8. wherein said fully enclosed 

curved surface is substantially spherical. 



15 
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10. Apparatus according to claim 5, wherein said surface and said 
transducers are remotely located from said processing means. 



11. A method of signal processing, in which an interaction at a 
5 spatial position produces an input signal, and said input signal is processed 
to produce an output signal indicative of said spatial position, characterised 
by the steps of: 

processing an active input signal to produce transformed data that 
conveys a characteristic of said input signal that is correlated to an active 
10 spatial position; 

comparing said transformed data with a plurality of stored data 
templates of known spatial position; 

producing a similarity score for each of said template comparisons; 
interpolating selected similarity scores to identify a value for said 
15 active spatial position; and 

producing an output signal in response to said interpolated position 

value. 



12. A method according to claim 11. wherein said spatial position 
20 is a distance from a transducer from which said input signal is derived. 



13. A method according to claim 11, including the step of 
identifying a plurality of said spatial positions in response to respective 
signals derived from a plurality of respective transducer means. 

14. A method according to claim 13, wherein a plurality of 
transducer means receive inputs form a common location and said location 
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is identified by combining said plurality of spatial positions. 

15. A method according to claim 11. wherein said transform data 
is generated by an identifying characteristic of frequency-related 
components of said signal. 

16. A method according to claim 11, wherein said active input 
signal is generated by transducing means in response to the sound of a 
movement made against a surface. 

17. A method according to claim 16, wherein said movement Is 
that of a user's fingertip. 

18. A method according to claim 16, wherein said movement is 
made against a curved surface. 

19. A method according to claim 15, wherein said movement is 
made against a fully enclosed curved surface. 

20- A method according to claim 19, wherein said fully enclosed 
curved surface is substantially spherical. 
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(57) Abstract: An apparatus is disclosed 
for supplying input signals to a computer. 
A sensor having the form of a sphere has 
a touch sensitive surface for generating 
position data for touch events. The sensor 
includes orientation sensors that determine 
rotation with respect to the earth's magnetic 
and gravitational fields. Orientation data 
may be combined with position data to 
interpret the orientation of touch events on 
the surface with respect to the computer's 
display. Cursor movement or text may be 
generated from touch events. Preferably 
the sphere has a roughened surface that 
generates sound when touched. Position 
data is generated by processing signals from 
microphones under the sphere's surface. 
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inputting Data 

Field of the Invention 

The present invention relates to an input apparatus for generating 
control signals for a computer. 

introduction to the Invention 

The tasks performed by the operator of a computer have defined the 
devices through which input data is generated. In the case of a personal 
computer, a keyboard and a mouse are used. 

Design and operation of computer peripherals maintains the difference 
between computers and other equipment such as radio, hi-fi and television. 
The requirement of a keyboard and mouse is becoming increasingly 
perceived as a major barrier to the wider use of computers in a much broader 
range of activities. 

An example is the emergence of MPS and other related audio 
compression standards. These provide high quality compression of audio 
data. Most radio stations around the worid are now able to broadcast over 
the intemet, in additional to their traditional location in the electromagnetic 
spectmm. Furthermore, it has become possible to store an entire CD 
collection on a low cost computer hard disk. However, the computer, in its 
present form, is not considered as a serious altemative to radio or hi-fi 
devices. A similar situation exists with video data. The preferred viewing 
device, except when editing, is a traditional television set. 

Computers are increasingly capable of receiving and manipulating 
many different media types within a common, easily used, computer- 
generated environment. However, the method of supplying input to the 
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computer terminal has restricted the wider use of this technology. The 
keyboard and mouse are best operated at a desk, and this prevents 
computers from being considered as replacements for a broad range of 
conventional electronic equipment. As computer and internet technology 
5 develops, increasingly the restrictions placed upon it are in the way the user 
interacts with computers through an input device. 

The digitisation of graphic design, video and film editing has led to the 
development of improved devices for interaction with image data. The most 
widely used of input devices In this context is the graphics tablet, which is 

10 operated in the manner of a pencil-with-paper. On a large graphics tablet, it is 
possible to provide an area having the function of a keyboard, and this may 
be operated to generate occasional text where this required. 

In three-dimensional computer modelling, no single preferred 
peripheral device has emerged. Several systems are known, optimised for 

15 particular applications. An example of this is radio tracking, which provides 
three dimensions of position and three dimensions of rotation. In a virtual 
reality application, a radio receiver is fixed to a users head-mounted display, 
and the position information obtained by analysing data from a fixed 
transmitter is used to determine stereoscopic images for the users eyes. The 

20 images are updated so as to provide an appropriate view for the angle and 
position of the user's head. Similar devices may be used to track the position 
of a hand, including devices that use ultrasound to determine orientation. 
Hand gestures resulting from finger movement may be tracked using a data 
glove. However, none of these devices is suitable for replacing a keyboard or 

25 mouse due to the requirement to suspend the devices in space in order to 
generate position data in the third dimension. 

Graphics tablets and three dimensional input devices may be suitable 
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for replacing the keyboard and mouse in certain applications. However, their 
high cost, and other practical considerations, make them unsuitable 
replacements for the widespread, low cost and ubiquitous keyboard and 
mouse of the personal computer. 

5 

Summary of the Invention 

It is an aim of the present invention to provide an improved input 
apparatus for supplying control signals to a computer. 

10 Brief Description of the Drawings 

Figure 1 shows a sensor and a computer terminal; 

Figures 2 and 3 detail construction of the sensor shown in Figure 1, 
including microphones and a sensor core; 

Figure 4 details components of the sensor core shown In Figure 3, 
15 including analogue to digital converters, orientation sensors and a digital 
signal processor; 

Figure 5 details the digital signal processor shown in Figure 4\ 

Figure 6 details one of the analogue to digital converters shown in 
Figure 4\ 

20 Figure 7 details a frequency domain analysis of audio data recorded 

from one of the microphones shown in Figure 2, in response to a first type of 
touch event; 

Figure 8 details a time domain analysis of audio data recorded from 
two of the microphones shown in Figure 2. in response to a second type of 
25 touch event; 

Figure 9 details steps executed on the digital signal processor shown 
in Figure 4, including a step of identifying a drag position, a step of identifying 
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a hit position and a step of identifying sphere orientation; 

Figure 10 details the step of identifying a drag position shown in 
Figure 9, including a step of con-elating with templates and a step of 
identifying intersecting arcs; 
5 Figure 1 1 details templates that are used in the step of correlating with 

templates shown in Figure 10\ 

Figure 12 illustrates the step of identifying intersecting arcs shown in 
Figure 10: 

Figure 13 details the step of identifying a hit position shown in Figure 

10 9; 

Figure 14 details the orientation sensors shown in Figure 4, including 
a magnetic field sensor and a gravitational field sensor, 

Figures 15 and 16 detail the magnetic field sensor shown in Figure 14\ 
Figures 17 and 18 detail the gravitational field sensor shown in Figure 

15 14] 

Figure 19 details the step of identifying sphere orientation shown in 
Figure 9; 

Figures 20 and 21 detail construction of a charger and receiver unit for 
use with the sensor and computer terminal shown in Figure 1\ 
20 Figure 22 details components of the computer terminal shown in 

Figure 7. including a memory; 

Figure 23 details contents of the computer memory shown in Figure 

22] 

Figure 24 summarises steps performed by the computer terminal 
25 shown in Figure 1, including a step of calibrating the sensor and a step of 
using the sensor; 

Figure 25 details the step of calibrating the sensor shown in Figure 24] 
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Figure 26 details the step of using the sensor shown in Figure 24, 
including a step of processing touch event data; and 

Figure 27 details the step of processing touch event data sho\Am in 

Figure 26. 



Detailed Description of The Preferred Embodiments 

A computer terminal is illustrated in Figure 1. The computer terminal 
101 comprises a high resolution display panel 102 and standard personal 
computer circuitry. The display 102 is the only visible part of the computer. 

10 - The components of the computer are built in to the display housing. The 
computer is connected to the intemet, and provides access to media of 
many different types, including audio, video, applications such as word 
processors, and so on. This highly varied functionality is provided by the 
combination of an intemet brovy^ser software application and a graphical 

15 user interface environment such as X-Windows. This combination of 
technologies, both hardware and software, with information presented 
through a graphical display, may be considered as a first kind of computer 
generated environment. A second kind of computer generated environment 
is one in which a three dimensional virtual world is presented on a two 

20 dimensional screen, and various forms of user input enable the user to 
navigate, as if physically present, in the virtual space. Virtual spaces of this 
type are presently used widely in games, but also for serious business 
applications, such as the representation of bug tracking in complex 
computer software development. Three dimensional virtual worids are 

25 widely believed to be the future of the Intemet, though at the present time, 
specific interpretations of this vision have not been sufficiently developed to 
provide practical engineering proposals. 
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In order to interact with a computer generated environment, a user 
must have convenient control over the position and orientation of their 
viewpoint. In the case of a conventional computer desktop environment, 
navigation involves convenient movement of a cursor or pointer 103. 
5 Furthermore, text entry is required also. The amount of text entry required 
depends upon the application, but these two requirements, movement and 
text input, can be seen as fundamental to any general purpose computer 
environment, of any type. 

The present day windows-based desktop environment was 

10 developed over many decades, and has evolved as a result of the 
availability of the mouse, along with a keyboard for text entry. At the time of 
their development, windows and mouse -based graphical user interface 
environments were not considered stressful to use. However, their use on 
every desktop, and near constant use daily in the workplace, has resulted 

15 in various stress-related symptoms, including Repetitive Strain Injury (RSI) 
and Carpal Tunnel Syndrome. That these injuries exist indicates that the 
conventional computer interface is at least mildly stressful to use, and that 
better methods of input will enable worker productivity to be increased as 
well as potentially opening up general purpose computers to increased 

20 daily use in non-stressful leisure-related activities. 

An improved computer input device is shown in Figure 1, The 
computer input device 104 is spherical in shape. The sphere is 
approximately six centimetres in diameter. Usually, the sphere 104 is 
supported in the left hand, as shown, while a finger of the right hand traces 

25 across its surface in order to affect and control the position of the pointer 
103 on the screen. The sphere 104 may be freely oriented in any degree, 
and movements of the finger are interpreted as being made with respect to 
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the screen 102, rather than locally with respect to the surface of the sphere 
104. In this way. the user can freely rotate and orient the sphere, while 
performing finger-movements on any part of its surface, which may then be 
interpreted as being performed in the space of the computer-generated 
5 environment, rather than in the physical environment of the input device 
104 Itself. This removes a layer of abstraction from the computer interface, 
and enables a language of interactions to be developed that is inherently 
intuitive and fast. 

In a two-dimensional desktop environment, the sphere presents an 

10 infinite surface over which the mouse pointer may be moved, thus 
mimicking mouse movements, but accomplished entirely with movements 
of the fingers, and including rotations of the sphere, or ball, 104, by the left 
hand when necessary. 

Deliberate movements of a finger across the surface are interpreted 

15 as a movement of the pointer in the computer-generated environment. 
Manipulations of the left hand generate touch event signals that can be 
ignored. The tracing of a finger across the surface of the sensor 104 is a 
first method of generating input data. Using a traditional computer mouse, a 
click or double click is used to signify an action that is performed with 

20 respect to the location of the pointer on the screen 102. This is achieved by 
a tap or double tap on the sensor 104, regarded as a hit. The position of the 
hit may be used to determine the nature of the command to be performed. 
However, a far wider variety of events is possible with the sphere 104. 

The sphere 104 may be used to enter text characters. To signify a 

25 change from the previously described method of graphical operation, the 
sensor is hit, fairly hard, anywhere on its surface. Thereafter, movements of 
a finger on the surface may be interpreted as alphanumeric characters. 
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Text entry may be performed one character at a time, using character 
recognition. Altematively, for high speed text entry, a form of shorthand 
writing may be used. 

The sphere 104 has a slightly roughened surface. Friction between a 
5 finger and the surface, as the fingertip is moved across it, results in the 
generation of random acoustic noise. The friction-generated noise is 
analysed in order to determine the position of a finger-drag event of this 
type. The arrangement of microphone sensors to perform detection of noise 
generated in this way is shown in Figure 2. The shell of the sphere 

10 comprises three millimetre thick silicone rubber, and this is covered by a 
fine nylon felt that provides a suitable noise-generating surface. Directly 
beneath the silicone rubber shell, arranged equidistant from each other, are 
four microphones 201, 202, 203 and 204. These have a frequency range 
substantially equal to that of human hearing, and can be general purpose 

15 audio microphones. Each microphone may be considered as being at the 
centre of a face of a three-sided pyramid, or regular tetrahedron. The angle 
made between any two microphones at the centre of the sphere is 
approximately one hundred and ten degrees. 

Signals received by an individual microphone may be analysed to 

20 identify the proximity of a noise-generating drag event, and the results of an 
analysis of this type from two or more microphones are combined to identify 
the location of the touch event on the surface of the sphere. 

Further details of the construcfion of the sphere 104 are shown in 
Figure 3. During manufacture, the silicone rubber shell is created in two 

25 halves 301 and 302. A central core 303 contains the circuitry of the sensor. 
Between the core 303 and the silicone rubber shell is a layer of acoustically 
isolating and shock absorbant polyester fibre 304. This construction firstly 
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ensures that microphones 201 to 204 are sufficiently acoustically isolated 
from each other, and that each microphone only receives sound directly 
from the silicone mbber shell. As a significant further advantage, however, 
this construction provides a very high level of shock-immunity, so that the 
5 sphere 104 may be handled extremely roughly without damage. The 
microphones 201 to 204 are embedded in moulded silicone rubber 
mountings in the shell itself. 

The two halves 301 and 302 of the sphere 104 are combined using 
an acoustically homogeneous silicone rubber seal. The core 303 contains a 

10 rechargeable battery, and this must receive power externally when 
recharging is required. In order to avoid compromising the structural 
integrity of the surface by a wire connection, an inductive loop 305 provides 
contactless access to the recharging power source. 

Circuitry contained within the central core 303 shown in Figure 3 is 

15 detailed in Figure 4. A first stereo analogue to digital converter (A-D) 401 
receives analogue audio signals from the first two microphones 201 and 
202 a second stereo analogue to digital converter 402 receives analogue 
audio signals from the second two microphones 203 and 204. A suitable 
analogue to digital converter is the CS53L32A, made by Cirrus Logic, 

20 available from http://www.cirrus.com. The converters 401 and 402 generate 
a multiplexed digital audio signal that is supplied to a digital signal 
processor (DSP) 403. The digital signal processor is preferably a Motorola 
DSP56603, that includes arithmetic and memory circuitry suitable for audio 
signal analysis. Design data and other information about the DSP56603 is 

25 available from http;//ebus.mot-sps.com. 

Analysis of signals supplied to converters 401 and 402 results in 
touch event position signals being generated, and these are transmitted 
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digitally from a transmitter 404 to a receiver connected to the computer 
terminal 101. Positional signals are generated with respect to the terminal 

101. irrespective of any degree of rotation of the sensor 104. This is 
facilitated by orientation sensors 405. The signals from the orientation 

5 sensors define the orientation of the sphere 104 with respect to the screen 

102, and thereby enable calculations to be performed that effectively 
remove the orientation of the sphere from the interpretation of touch events 
made upon its surface. Said touch events are thereafter considered and 
interpreted with respect to the screen, regardless of the sphere's actual 

10 orientation. From the user's point of view, the sphere always looks and 
feels the same, and can rotate it and operate it without concem for its 
orientation. 

In order to measure orientation of the sphere, the orientation sensors 
405 characterise sensor orientation as a first rotation RM about a horizontal 

15 axis due to the Earth's magnetic field and a second rotation RG about a 
vertical axis due to the Earth's gravitational field. By combining these data 
items with data about the positions of touch events occurring on the surface 
of the sphere, it is possible to interpret user touch events on the sphere 
with reference to a standard space in which the computer terminal 101 is 

20 located. Thus, for example, a forward dragging movement of the finger 
towards the screen 101 will move the cursor upwards, regardless of the 
sphere orientation. 

The core 303 includes a power management circuit 406, that 
facilitates low power and shutdown modes for the DSP 403, and other 

25 circuitry. The power manager 406 receives power from a rechargeable 
NiMH battery 407, and also facilitates rectification and current regulation of 
recharging power supplied from the inductive loop 305. 
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The digital signal processor 403 shown in Figure 4 is detailed in 
Figure 5. Several data and address busses are present within the DSP, and 
these are summarised for the sake of clarity by a simplified wiring 
connection 501. Timers 502 provide pulse width timing capabilities that are 
used to measure signals from the orientation sensors 405. Input and output 
circuits (I/O) 503 provide several physical connections from the DSP to the 
other components in the core, including the A-D converters 401 and 402. 
The program ROM and RAM 504 includes bootstrap instructions and 
control instructions for co-ordinating interface operations with other circuitry 
in the core 303. and for performing real time signal analysis. An X data 
RAM and a Y data RAM 505 and 506 provide a pair of operands per 
instruction cycle to an arithmetic and logic unit (ALU) 507. The ALU is 
thereby capable of fetching and multiplying two data operands from X and 
Y memory 505 and 506 in every instruction cycle. This arrangement 
facilitates efficient implementation of the processing algorithms that are 
required in order to determine the position of touch events on the surface of 
the sphere. 

Each of the A-D circuits 401 and 402 comprise circuitry as shown in 
Figure 6. A first channel pre-amplifier 601 receives an unamplified audio 
signal from a microphone and increases its intensity to that which is 
suitable for digital to analogue conversion. An anti-alias filter 602 removes 
frequency components above half the sampling rate, so as to ensure that 
subsequent frequency analysis of audio data gives an accurate 
representation of the spectrum. The output from filter 602 is supplied to the 
left input of a stereo sixteen bit low power analogue to digital converter chip 
603. The sampling rate is 44.1kHz. Another channel is implemented for a 
second microphone using pre-amplifier 604 and anti-alias filter 605. A 
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common multiplexed output is supplied from the A-D converter chip to the 
DSP 403. Additional clock and word synchronisation signals have been 
omitted from this diagram for the sake of clarity. 

An analysis of several seconds of audio data from a single channel 
5 is shown in Figure 7. The vertical axis represents time, and the vertical axis 
represents frequency. In this graph, the amplitude of a particular frequency 
component at a particular time is represented by density. The graph shows 
a plot of a signal that results from the movement of a finger across the 
surface of the sphere. 

10 At the start of the plot 701 the fingertip is distant from the 

microphone. Sound waves reaching the microphone are primarily 
transverse waves, oscillating perpendicular to their direction of propagation. 
The silicone mbber filters high frequencies but has little effect on the low 
frequencies. Thus, the signal reaching the microphone, regardless of its 

15 actual amplitude, contains an indication of the distance due to the relative 
strengths of high and low frequencies. As the finger tip moves closer, 
higher frequencies increase, while the strength of the low ones remains 
substantially the same. As the fingertip becomes increasingly dose to the 
microphone, at 703, a completely new set of frequencies is added in the 

20 spectrum. This is due to longitudinal waves being transferred across the 
thickness of the silicone aibber from the fingertip, directly to the 
microphone. As the microphone is approached, there is a mixture of both 
longitudinal and transverse waves, as identified from the two distinct areas 
of the graph at 703 and 702. 

25 A final exceptional condition is reached when the fingertip is directly 

over the microphone. The high frequency components are generated by the 
friction between the edge of the finger and the roughened surface of the 
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sphere. However, when the finger is directly over the microphone, these 
high frequency components are masked, and the sound picked up by the 
microphone comes from the centre of an area of the fingertip alone. At 704, 
a sudden loss of high frequencies occurs because these frequencies are 
5 damped by the area of the fingertip. The low frequencies exhibit a 
characteristic change, having a more balanced profile. 

These changes provide a characteristic set of descriptions for finger 
dragging events that occur on the surface of the sphere. By comparing the 
outputs two or three channels at once, the position of a moving fingertip 

10 anywhere on the surface of the sphere 104 may be identified. 

Hitting the sphere, to perform the equivalent of a mouse click, does 
not contain as much frequency data, and so a different type of analysis is 
used. The speed of transverse sound waves in the silicone rubber shell is in 
the order of only twenty metres per second. This makes it possible to 

15 discern a time difference for wavefronts arriving at different microphones. A 
pair of graphs resulting from a simultaneously digitised hit event are shown 
in Figure 8, Trace 801 is for the more distant microphone, and it can be 
seen that this commences a short period after the second trace 802. The 
difference in initial characteristic wavefronts is in the order of two 

20 thousandths of a second. This provides a reasonably accurate source of 
position data. The traces 801 and 802 also exhibit differences in frequency 
content, which may be observed in the jaggedness of the second trace 802, 
which is the microphone nearest to the hit event Waveform 801 reaches a 
higher peak, due to the lack of damping provided by the finger over the 

25 point of impact just after the hit event has occurred. Several such 
characteristics may be analysed and the results combined so as to identify 
a touch event characteristic to an increased level of accuracy. 
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The main sequence of steps performed by the DSP 403 shown in 
Figure 4 is summarised in the flow chart in Figure 9. At step 901 a 
frequency domain analysis is performed on each of the four channels of 
buffered audio data. At step 902 a question is asked as to whether a drag 
5 or a hit event has been observed in the data. It is possible that neither is 
identified, either due to lack of occurrence, or because a clear characteristic 
cannot be identified. If neither drag nor hit is present in the audio data, 
control is directed to step 905. If a hit is obsen/ed, control is directed to step 
904. or, if a drag is observed, control is directed to step 903. At step 903 

10 the drag position is identified. At step 904 the hit position is identified. 

At step 905 the sphere orientation is identified by analysing data 
from the orientation sensors 405. At step 906 a question is asked as to 
whether any data needs to be transmitted to the computer. For example, if 
no touch event has occurred, and the orientation has not changed, no data 

15 need be transmitted, thus saving battery life. If no events occur over a 
prolonged period of time, say twenty seconds, the sensor can be placed in 
a power down mode. When held in either hand, even if not being used, the 
sensor will sense small changes in orientation, indicating that it is probably 
about to be used. If data is available for transmission, control is directed to 

20 step 907, where data is transferred over a serial link from the DSP 403 to 
the radio transmitter 404. for transmission to the computer 101 . 

The step of identifying the drag position 903, shown in Figure 9. is 
detailed in Figure 10, At step 1001 the four channels of buffered audio data 
are analysed to identify the three with the greatest amplitude. These three 

25 are the channels whose analysis will yield the most accurate touch event 
characterisation. The three loudest channels are called A, B and C. At step 
1002 the first of these three channels is selected. At step 1003 a frequency 
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domain analysis is perfomned, and the results of this are normalised, such 
that the loudest frequency component has an amplitude of one. At step 
1003 a correlation is performed with respect to a set of templates. Each 
template characterises a particular frequency response that is expected to 
5 occur at a known distance from a microphone. Thus, with reference to 
Figure 7, a template exists for the frequency characteristic at 701, 702, 703 
and 704. Each template has a different shape. The degree to which actual 
microphone data matches one of these templates indicates its proximity to 
the characteristic distance of that template. 

10 A correlation score is generated as a result of step 1004, and at step 

1005 the two best scoring templates are selected. It is then known that the 
actual distance of the event from the microphone for that channel is 
between the characteristic distances of these two templates. At step 1006 
the actual distance is identified by interpolating between the two 

15 characteristic distances, in proportion to the difference between the 
template scores. This identifies the characteristic distance for the channel, 
which may be DA, DB or DC, depending on which channel is being 
analysed. At step 1007 a question is asked as to whether there is another 
channel remaining to be analysed. If so, control is directed back to step 

20 1002. Altemativeiy, each distance DA. DB and DC will have been identified. 
At step 1008 intersecting arcs are identified across the surface of the 
sphere for each characteristic distance, and at step 1009 a point. P. is 
identified that is defined by the nearest point of convergence for the three 
arcs defined by DA, DB and DC. 

25 The templates that are used in step 1004 in Figure 10 are illustrated 

in Figure 11. Template 1101 corresponds to an ideal frequency response at 
a distance of D=40 mm away from a microphone. The signal at 701 in 



wo 01/43063 PCT/GBOO/04635 



16 

Figure 7 would closely match this template. Template 1102 has a 
characteristic distance D=25 mm, and roughly con^esponds to the plot at 
position 702 in Figure 7. Template 1103 has a characteristic distance D=10 
mm, and would provide a high score for a signal occurring just after point 
5 703 in Figure 7. Template 1104 corresponds to the fingertip being directly 
over the microphone, and corresponds to point 704 in Figure 7. By 
selecting the best two corresponding templates, the actual distance of the 
event from the microphone may be identified by interpolation between 
characteristic distances of the templates. 

10 The distance of an event may be considered as a notional distance, 

as frequency characteristics may change for different finger sizes, applied 
pressure and other variable factors. Whatever the distances are, DA, DB 
and DC define three characteristic arcs, whose ideal convergence point P is 
illustrated in Figure 12, The ideal convergence point is the same regardless 

15 of these variable factors. 

Identification of a hit position, shown at step 904 in Figure 9, is 
detailed in Figure 13, At step 1301 the three loudest channels A, B and C 
are identified. At step 1302 the first of these channels is selected for 
analysis. At step 1303 the signal is filtered. The filter removes frequencies 

20 below two hundred and fifty Hz. as this results in a better analysis being 
performed. Preferably an FIR linear phase filter is used. However, an MR 
filter, such as that used to generate the trace shown in Figure 8, is 
acceptable, with a slightly reduced accuracy of results. At step 1304 an 
event start time is identified by analysing the channel data. At step 1305 a 

25 question is asked as to whether another channel remains to be analysed, 
and if so, control is directed back to step 1302. Alternatively, start times will 
have been identified for each of channels A, B and C, and control is 
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directed to step 1306. 

The difference between start times for a pair of channels identifies a 
distance from the mid point between two microphones. On the surface of 
the sphere, this mid point is expressed as a line. At step 1306 distances are 
5 identified for each combination of start times, resulting in three lines, or 
arcs, being identified across the surface of the sphere. At step 1307 a 
characteristic common point is identified, in a similar manner to that shown 
in Figure 12, In theory, two such arcs are required. However, three are 
used to improve accuracy. Four may be used, if the channel data for all four 

10 microphones is of sufficiently high quality. 

The invention provides a method of generating positional information 
by analysing the noise generated by friction between an object and a 
surface. Preferably the friction is generated by the movement of a finger 
across a surface. The surface may be flat, or curved, regulariy or 

15 irregulariy, or spherical. The surface may be elastic, temporarily or 
permanently deformable under finger pressure, or rigid. The invention 
includes any kind of interaction between a user's fingers and any such 
surface where resulting sounds are analysed to generate position 
information for a computer-generated environment. Position information 

20 generated in this way may be combined with information that defines the 
orientation of the surface with the projection upon a display device of a 
computer generated environment. This combination of position and 
orientation information enables sound-generating touch events made upon 
the surface to be interpreted with respect to a computer environment, 

25 in^espective of the orientation of the surface. 

Orientation of the sensor is defined by rotations RM and RG about 
Earth's magnetic and gravitational fields. Detail of the orientation sensors 
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405 shown in Figure 4 is shown in Figure 14, fK magnetic field sensor 1401 
and a gravity field sensor 1402 generate digital oscillation signals whose 
periods are measured in order to ascertain orientations within respective 
fields. A multiplexer and counter circuit 1403 provides interfacing and 
5 control signals for the oscillating circuits, and divides the frequencies down 
by an amount suitable for highly accurate measurement by the timers 502 
in the DSP 403. 

The magnetic field sensor 1401 is detailed in Figures 15 and 16. 
Figure 15 shows three mutually orthogonal inductors. Each inductor is less 

10 than ten millimetres long. Figure 16 details a circuit suitable for detecting 
the polarity and magnitude of the Earth's magnetic field with respect to 
each of the three inductors. A logic gate 1601 provides a positive or 
negative DC bias via resistor 1602 to the presently selected inductor 1501 
to 1503. Selection of an inductor is provided by logical control signals 

15 supplied to tri-state logic buffers 1602, 1603 and 1604. An operational 
amplifier 1605 provides amplification for sustaining oscillations whose 
frequency is determined by the inductance of the selected inductor 1501 to 
1503. The DC bias provided by resistor 1602 drives the core of the selected 
inductor to near saturation. 

20 Near to saturation, a coil's inductance changes in response to the 

applied field, even though the coil's windings and core are fixed. The 
addifional offset towards or away from saturation, resulting from the Earth's 
magnetic field, may be detected in this way. By switching polarity of the DC 
bias in the coil, it is possible to determine the polarity of the Earth's 

25 magnetic field when different resulting oscillation frequencies are 
compared. If there is no difference, this indicates that the coil is aligned 
orthogonally to the Earth's magnetic field. The output from the operational 
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amplifier 1605 is supplied as a logic signal to the counter 1403, and the 
DSP 403 determines the precise frequency of oscillation for each of the 
coils, in each polarity, and thereby the alignment, in three dimensions, of 
magnetic North. The circuit requires a couple of milliamps of current to 
5 operate, and the sensors are extremely small and of low cost. A suitable 
inductor, of the type shown in Figure 15, is the SEN-M magneto-inductive 
sensor, available from Precision Navigation of Menio Park. California. Use 
of partly saturating inductors to detect the Earth's magnetic field in this way 
is detailed in United States patent 4.851 ,775. 

10 The gravity field sensor 1402. shown in Figure 14, is detailed in 

Figures 17 and 18, An enclosed spherical container is half filled with a liquid 
having a substantially different relative permeability to that of free space, at 
a frequency of around five hundred kHz. A suitable liquid is mercury, which 
has a relative permeability of around 0.7 at this frequency. Three coils 

15 1702. 1703 and 1704 are wound in close proximity to the container 1701. 
and are mutually orthogonal. Each coil is connected in an oscillator circuit 
as shown in Figure 18, Each coil forms the inductive part of a tuned circuit, 
that also comprises two capacitors. A logical HCMOS NOR gate 1801 to 
1803 provides amplification and a selection input to activate the oscillator 

20 circuits separately. The outputs from the oscillators are combined in a 
three-input NOR gate 1804, so that an inductor selected for oscillation by a 
logical high input to gate 1801, 1802 or 1803. has its characteristic 
frequency presented as a square wave at the output of gate 1804. The 
output from gate 1804 is supplied to the counter circuit 1403. The 

25 frequencies of oscillation of the three coils depend upon the amount of 
immediately adjacent mercury. The three frequencies are measured in the 
DSP 403, and interpolated look-up tables are used to determine the actual 
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orientation of the sensor with respect to the Earth's gravitational field. The 
circuit of Figure 18 requires less than one milliamp to operate. 

The three-dimensional vectors for magnetic and gravitational 
ambient fields may be represented as two rotations of the sphere, RM and 
5 RG, about the magnetic North axis, and about the vertical gravitational axis. 

The process of identifying RM and RG combines measurements 
from both the gravitational and magnetic field sensors. It is possible for 
either to experience interference, and the gravitational field sensor 1402 
may experience instability due to motion of the mercury in the container 

10 1701, particulariy if the sensor is moved about rapidly. The step of 
identifying the orientation of the sphere 905, shown in Figure 9, is detailed 
in Figure 19, At step 1901 the rotation about magnetic North, RM, is 
identified. At step 1902 Kalman filtering is applied to the value of RM. A 
Kalman filter determines a measure of confidence in the current 

15 measurement, and increases low pass filtering when this confidence value 
is low. At step 1903 the rotation RG of the sphere about the vertical axis 
due to gravity is determined, and at step 1904 Kalman filtering is also 
applied to this value. At step 1905, filtered values for RM and RG are stored 
for later transmission to the computer 101 when necessary. 

20 In an alternative embodiment the Earth's magnetic field can be 

detected using silicon sensors employing the giant magneto-resistive effect, 
in a further alternative embodiment the Earth's gravitational field can be 
detected using accelerometers, which are more expensive, but facilitate 
detection of rapid acceleration of the ball 104, which may be used as a 

25 further source of gestural infomriation for navigating a computer 
environment. 

After an extended period of use, the battery 407 requires recharging. 
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In order to recharge, the inductive loop 305 must be placed in an adjacent 
position to another matching inductive loop that is supplied by a IkHz 
power source. This may be located in a charging unit as illustrated in Figure 
20, The outside of the sphere has a mark directly opposite the location of 
5 the inductive loop 305, and this mark must be uppermost when the sphere 
is placed on the recharging unit 2001. The recharger 2001 may also 
conveniently double as the receiver for the data from the transmitter circuit 
404, and a serial connection 2002 provides the connection for this data to 
the computer terminal 101. 

10 Details of the charger and receiver unit 2001 are shown in Figure 21. 

An inductive loop and oscillator 2101 supply an altemating magnetic field to 
the inductive loop 305 in the sensor 104 during recharging. During use, the 
sensor 104 transmits radio signals to a radio receiver 2102. A central 
processing unit (CPU) 2103 provides error correction of data received over 

15 the radio link. A universal serial bus (USB) interface 2104 provides a 
connection to the computer terminal 101 via the serial cable 2002. 

The computer terminal 101 shown in Figure 1 is detailed in Figure 2. 
A central processing unit (CPU) 2203 provides co-ordination and 
processing for the terminal 101. Instructions and data for the CPU 2203 are 

20 stored in main memory 2204, and a hard disk storage unit 2205 facilitates 
non-volatile storage of data and several software applications. A modem 
2206 provides a connection to the internet. A universal serial bus (USB) 
interface 2207 facilitates connection to the charger and receiver unit 2001. 
Touch event and orientation data are received from the sphere 104 via the 

25 USB interface 2207. A graphical processor 2208 provides dedicated 
graphics rendering capabilities to speed up the display of high resolution 
graphical images on the display 102. An audio processor 2209 supplies 
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audio signals to loudspeakers in the computer terminal 101, and receives 
audio signals from a microphone 2211. 

The contents of the main memory 2204 shown in Figure 22 are 
detailed in Figure 23, An operating system provides common functionality 
5 for software applications 2302. A device driver 2303 for the sensor 104 is 
also stored in main memory 2204 while the computer terminal 101 is 
switched on. The sequence of operations necessary to operate the sensor 
104 is detailed in Figure 24. At step 2401 the sensor 104 is charged using 
the charger and receiver unit 2001 shown in Figures 20 and 21, At step 
10 2402 the sensor is calibrated. In order to use the sensor, it is necessary to 
store orientation data so that the device driver 2303 is able to determine 
which way is forward, backwards, left and right with respect to the Earth's 
magnetic field, and therefore also substantially with respect to the terminal 
101. 

15 It is assumed herein that orientation of the sensor within the Earth's 

magnetic and gravitational fields effectively provides orientation with 
respect to the terminal 101. Cleariy this would be untrue if the user 
operated the sensor from behind the terminal. However, for the purposes of 
practically operating the sensor, it may be assumed that touch gestures on 

20 the surface of the sphere 104 that are made with respect to the Earth's 
magnetic and gravitational fields are also made with respect to the location 
of the computer terminal 101. If the terminal position is changed 
substantially, it will be necessary to perform the calibration at step 2402 
again. At step 2403 the sensor 104 is used, and at step 2404 the sensor 

25 1 04 is recharged. The design of the charger and receiver unit 2001 is such 
that the sensor 104 may be conveniently left at rest on the charger 2001 
whenever it is not in use. 
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T.e step of calibrating *e sensor 2402. shown in Rgur. 24 is 
r » 25 A. step 2501 the computer requests the user to drag 

left to nght orientation data from 

26 A. lep 2601 touch event and or ohentation data is receded fron, 

' Z Z ^n*<.es .he angles of n,.a«on RM and RG a.>ou. the 
me sensor 104. Th,s ,nciu ^ 

2604. At s p ,„,„ sG and RM, resulting in touch event data 

^„ oppos^a and egual degree to SG and ^ ^^^^^^^^ ^ 

that has a stationary co-ordinate system, irrespective 
r ere A. step 2605 ^ resuming touch event data Is processed, 
the sphere. At step 

Finally, at step 2606 the tenmnal-onented data PP 

system 2301 via the device dnver 2303, for use by apphcabons 2302. 

The step of processing touch even, data 2605. shown ,n F^ure 26. 
I 27 At step 2701 a question is asKed as to whether a 
. detailed in f^ure 27. ^^^^ " 3,,^ ,^3 f,.m all four 

larae nit has been received. A large hit is one wn 

. « extremely loud, indicating the user has hit the 

microphone channels ,s extremely . 

sensor quite hard, request a change of mode. 1. a la-^e 
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control is directed to step 2702 where the current mode is swapped. 
Alternatively, if a large hit is not identified at step 2701, control is directed to 
step 2703. At step 2703 a question is asked as to whether the currently 
selected mode for the sensor is graphics mode or text mode. If text mode is 
5 selected, control is directed to step 2706. Alternatively, graphics mode is 
identified, and control is directed to step 2704. At step 2704 any small hits 
are interpreted as the equivalent of mouse button clicks. Finger drag events 
are used to modify the X and Y co-ordinates of the cursor. At step 2705 the 
cursor position is updated. 

10 When used in text mode, control is directed from step 2703 to step 

2706. At step 2706 any small hits are interpreted as the equivalent of 
CAPS, CTRL and SHIFT events on a conventional keyboard. At step 2707 
touch movements on the surface of the sphere are interpreted as character 
entry events. Altematively, for high speed text entry, a form of shorthand 

15 can be used. 

In an alternative embodiment, the DSP 403 performs data 
compression of audio signals from the microphones 201 to 204. The 
compressed audio data is combined with orientation data, and is 
transmitted to the computer terminal 101 for analysis to determine surface 

20 event characteristics. 

in the embodiments described above, the sensor is a passive 
device, requiring a sound to be made by a touch event on the surface of the 
sphere, in an alternative embodiment, an active sensor is provided. Sound 
may be injected into the surface of the sphere, and a surface pressure map 

25 may be constructed from sound characteristics that result from interference 
and reflection. If the sensor shell is made of a hard material, ultrasonic 
sound may be used to generate a highly detailed pressure map. The 
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pressure data may be used to facilitate additional methods of data entry. An 
alternative method of detecting pressure is the use of multiplexed pressure 
sensitive electrical sensors, whose conductivity changes in accordance with 
the applied pressure. 

In a further alternative embodiment, touch events are detected by an 
electrical sensor in which capacitance of electrical conductors, near or 
embedded in the sphere's surface, is modified by the presence of parts of a 
hand or finger. 

The spherical shape of the sensor facilitates rotation in any degree 
without this making any difference to the appearance or feel presented to 
the user. Touch events on the surface are made with respect to the 
computer display 102. This has the psychological effect of extending the 
computer generated environment out into the space between the user and 
the terminal. 

Although in the preferred embodiment the entire surface of the 
sphere is touch sensitive, it is possible that, for the purpose of providing a 
direct electrical connection during recharging, that a different embodiment 
may include an insignificant portion of its surface where touch sensing is 
not fully provided. Also, in certain embodiments, the spherical shape may 
be distorted, as a result of squeezing or due to a preferred distorted shape. 

The sphere, being the simplest of three dimensional shapes, 
provides a suitable shape for universal object mapping. A complex shape, 
such as a telephone handset, may have its surface mapped to the surface 
of the spherical sensor 104, and interaction with the handset, for example 
to dial a number, may be effected via interactions with the sphere, 
Altematively, the shape of a three-dimensional object mapped in such a 
way may be modified using the touch events on the sensor's surface. The 
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shape, then changed, is remapped to the surface of the sphere, so that 
additional changes may be made. The universal shape of the sphere lends 
itself to interaction with a rich variety of complex shapes and forms. 
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Claims 

1. Input apparatus for a data processing system having the form 
of a fuiiy enclosed substantially spherical object, including touch-sensing 

5 means for generating position signals indicative of the position of touched 
events occumng anywhere on the surface of the object; 

orientation sensing means for identifying the orientation of the object; 
processing means for combining touch data with orientation data; 

and 

10 transmitting means arranged to transmit processed data to a data 

processing system. 

2. Apparatus according to claim 1, wherein said processed data 
represents said touch event signals orientated with respect to said display 

15 apparatus, by making reference to said orientation signals. 

3. Apparatus according to claim 2, wherein said orientation 
sensing means includes an ambient magnetic field sensor. 

20 4, Apparatus according to claim 3. wherein said magnetic field 

sensor comprises three mutually orthogonal magnetic field detectors. 

5. Apparatus according to claim 2, wherein said orientation 
sensing means includes gravitational sensing means. 

25 

6. Apparatus according to claim 5, wherein said gravitational 
sensing means comprises 
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a mobile inductor core in an enclosure, and 

three mutually orthogonal inductors responsive to the position of said 
mobile core. 

7. Apparatus according to claim 6, wherein a said inductor is 
included in an oscillating circuit. 

8. Apparatus according to claim 7. including counting means for 
measuring a frequency generated by said oscillating circuit 

9. Apparatus according to claim 6, including 
means for measuring an inductance of a said inductor, 
processing means for processing said measurement, wherein 

said processing means includes look-up means for identifying the 
orientation of said sphere with respect to gravity. 

10. Apparatus according to claim 1, wherein said sensing means 
comprises a plurality of sound transducer means. 

11. Apparatus according to claim 10, wherein said sphere has a 
surface that generates sound in response to a touch event. 

12. Apparatus according to claim 11, wherein said sound 
generating surface generates a noise-like sound in response to a dragging 
motion of a finger. 

13. Apparatus according to claim 11, wherein said sensing means 
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comprises a plurality of microphones. 

14. Apparatus according to claim 13, including processing means 
arranged to compare signals from said microphones in order to identify a 

5 characteristic of a touch event. 

15. Apparatus according to claim 14. wherein said characteristic 
is a position. 

10 16. input apparatus for a computer with a graphic display means, 

having the fonm of a fully enclosed sphere, including 

touch sensing means for generating position signals indicative of the 
position of touch events occurring anywhere on said spherical surface, 

transmitting means arranged to transmit signals to a computer in 
15 response to said position signals. 

orientation detecting means for detecting orientation of said sphere, 

and 

processing means configured to combine signals from said touch 
sensing means and said orientation detecting means for subsequent 
20 identification of a touch event orientation substantially with respect to said 
graphic display means. 

17. Apparatus according to claim 16, wherein said touch sensing 
means comprises a plurality of microphones and said sphere has a surface 

25 that generates sound in response to touch events. 

18. Apparatus according to claim 17, wherein said sphere has a 
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roughened surface. 

19. Apparatus according to claim 16, wherein said computer 
includes processing means configurable to perform steps of: 

selecting a graphical or text mode for said sensor. 

when in said graphical mode to perform steps for identifying a 
graphical instruction in response to touch events, and 

when in said text mode to perform steps for identifying text in 
response to touch events. 

20. Apparatus according to claim 16, wherein said sensor 
includes rechargeable cell means and a recharging inductive loop means. 

21. A method of generating control signals for a data processing 
systems displaying a graphical interface on a display apparatus, wherein 

a sensor has a substantially spherical form and includes: 

an outer surface and touch sensing means for generating position 

signals indicative of the position of touch events on said surface; 
orientation sensing means for identifying orientation; and 
transmitting means for transmitting sensed data to a data processing 

system, comprising steps of 

identifying the position of a touch event on the surface of said 

sensor; 

identifying the orientation of said sensor; and 

combining data representation position with data representing said 
orientation to generate such event signals orientated substantially with 
respect to display apparatus. 
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22. A method according to claim 21, wherein said touch sensing 
means includes acoustic transducer means, and said position signals are 
generated by processing sound signals. 

5 

23. A method according to claim 22, wherein said sound 
processing includes frequency domain analysis. 

24. A method according to claim 23, including identifying probable 
10 distances from sensors of a touch event in response to frequency 

characteristics of sounds at said sensors. 

25. A method according to claim 24, wherein said distances are 
combined to identify a location. 

15 

26. A method according to claim 25. wherein additional 
processing of said signals is performed in order to reduce positional error. 

27. A method according to claim 23, including measuring a time 
20 interval between the start of a sound at a plurality of acoustic transducer 

means. 

28. A method according to claim 27, wherein said sound is 
generated in response to a hit event on the surface of said sphere. 

25 

29. A method according to claim 23, wherein frequency analysis 
is used to identify a characteristic of drag touch events and start-time 
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analysis is used to identify a characteristic of hit touch events. 

30. A method of interacting with a three dimensional object model 
in a computer generated environment, wherein signals are supplied to said 

5 environment from a surface touch position sensitive sphere, comprising 
steps of: 

generating a mapping from said sphere to said object; and 
receiving touch events from said sphere and interpreting them to 
interact with a respective surface portion of said object. 

10 

31. A method of modifying a three dimensional object model in a 
computer generated environment, wherein signals are supplied to said 
environment from a surface touch position sensitive sphere, comprising 
steps of: 

15 generating a mapping from said sphere to said object; 

receiving touch events from said sphere and interpreting them to 
manipulate a respective surface portion of said object; 
updating said object; and 

generating a new mapping from said sphere to said object. 

20 

32. A method of generating positional input signals for a 
computer, in which the position of an interaction between a moving object 
and a suri'ace is identified by an analysis of the sound generated by friction 
between said surface and said object. 

25 

33. A method according to claim 32, wherein said object is a 
computer user's finger. 
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34. A method of supplying positional infonnation to a computer, in 
which the position of an interaction between a user's finger and a surface 
generates sound by friction beftveen said surface and said finger; 

5 said sound is converted into an electrical signal by acoustic 

transducer means; and 

signals derived from said sound are transmitted to a computer such 
that the position of said interaction controls a position characteristic in an 
environment generated by said computer. 

0 

35. Apparatus substantially as herein described with reference to 
Figures 1,2,3 and 4. 



15 



36. A method substantially as herein described with reference to 
Figures 1,2,3 and 4. 
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3. Claims: 10-15,17,18,22-29,32-34 

input device with sound actuated touch detectors 

4. Claim : 19 

input device connected to a computer switching between text 
and graphical mode 

5. Claim : 20 

input device with rechargeable cell 

6. Claims: 30,31 

spherical touch input device mapped to a three dimentional 
object 
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^_ Intemationai Application No. PCTX5B 00 )04635 

FURTHER INFORMATION CONTINUED FROM PCT/ISA/ £10 



Continuation of Box 1.2 
Claims Nos.: 35,36 



Rule 6, 2(a) PCT: Claim refering to other parts of the application 

The applicant's attention is drawn to the fact that claims, or parts of 
claims, relating to inventions in respect of which no international 
search report has been established need not be the subject of an 
international preliminary examination (Rule 66.1(e) PCT). The applicant 
is advised that the ERG policy when acting as an International 
Preliminary Examining Authority is normally not to carry out a 
preliminary examination on matter which has not been searched. This is 
the case irrespective of whether or not the claims are amended following 
receipt of the search report or during any Chapter II procedure. 



